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To assist in the development of policies to eliminate 
the subtle inequities which inhibit the mathematics achievement of 
many Black youths, it is suggested that the redesign of the National 
Assessment of Educational Progress (NAEP) mathematics assessment 
study Black students' opportunity to learn mathematics. 
Recommendations for the redesign of the NAEP mathematics assessment 
and its reporting formats are based on a cross-sectional study of 
mathematics achievement data on 14,289 Black, White^ and Hispanic 
17-year-old students from the third NAEP assessment. Achievement on 
the algebra or pre-algebra subtests was evaluated against mathematics 
course enrollment. Other variables studied included grade level, 
school racial composition, curriculum type, achievement level, race 
or ethnicity, and affective responses. Results indicated significant 
racial and ethnic differences in enrollment by curriculum (general 
versus academic) and in achievement. Schools* racial composition also 
appeared to affect achievement. Preliminary test item analysis 
suggests the need to use additional criteria in determining item biciS 
for Blacks. Because it is a valuable database for research, it is 
strongly recommended that NAEP broaden its scope to include measures 
of the opportunity to learn mathematics as a standard variable and to 
report school achievement impacted by school culture, classroom 
processes, and significant others. It is further recommended that 
NAEP restructure cognitive items on which Black students perform 
poorly to e:ftplore the causes for these response patterns. Changes in 
NAEP*s affective and background measures are also proposed. (GDC) 
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Proposals for Improving the NAEP Mathematics Assessment 

of Black Youth 



Introduction 

The reconstruction of the National Assessment of Educational Progress 
(nAEP) mathematics assessment to allov/ for substantial educational policy 
research is critical to enhancing opportunities for Black youth to achieve in 
mathematics. Findings of what all (most, and some) students know in 
raathoraatics and changes in the extent of this knowledge by cognitive process 
levels over time are well documented in traditional "NAEP Report Cards" to 
the nation. However, these reports cards do not yield results from which 
policy can be directed to eliminate the subtle inequities which inhibit 
the mathematics achievement of large numbers of Black youth. 

To achieve this end, equity issues must be incorporated into the NAEP 
design. The aim is neither to politicize the NAEP, nor to promote increased 
scores for Blacks at the expense of other students. Conversely, the aim is to 
ensure that achievement gains (losses) are reported as a function of changes 
in opportunity to learn. Accordingly, the purpose of this paper is to create 
the awareness of the nsed for NAEP to identify, clarify, obtain measures 
(certainly to be crude measures initially), and provide reports to the nation 
of the extent of the opportunity Blacks have to learn the mathematics content 
on which they are being assessed. 

This paper is not concerned with how educators, policy makers, test 
directors, and others implement these results. Rather, the scope of this paper 
is limited to proposals for the redesign of the NAEP mathematics assessment 
and its reporting formats. These recommendations are based on this author's 
experiences in completing the study, The Effect of Mathematics Course 
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Enrollment on Racial/Ethnic Differences In Secondary School Mathematics 
Achievement (1986), conducted while she was a visiting scholar at the NAEP 
Center at the Educational Testing Service (ETS), Princeton, New Jersey. This 
cross-sectional, atypical study of NAEP data focv.sed exclusively on 
racial/ethnic differences In mathematics achi'^vement which occurred during one 
assessment year, the third assessment ( data from the fourth assessment are 
not yet In the public domain). 

Highlights of The Research Stud y 
Methodology and Variable Identification 

Seventeen-year-old students who participated In the NAEP third 
mathematics assessment constituted a nationally representative sample of 
14,289 students of whom 76% were white, 16% were Black, 6% were Hispanic, and 
2% were "other race." In that this study focused on black, white, and Hispanic 
students, the latter group ^-las excluded from consideration. 

NAEP cognitive items in Booklets 7 through 13 were classified as 
pre-algebra or algebra depending on the content and indicated skills. 
Achievement on these item sets was evaluated against the respondents' 
self-reported mathematics training. The primary independent variable, then^ 
was "Course Enrollmpnt." For ease of analysis, course sequences were labeled 
as follows (students were assigned according to whether they indicated 
having studied each course in the sequence for at least one- half year)^ 
A - Algebra 1, geometry. Algebra 2, trigonometry; 
B - Algebra 1, geometry. Algebra 2; 
C - General Business, Algebra 1, geometry; 
D - Algebra 1, geometry; 
E - General Business, Algebr^J^ I; 
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F - Algebra 1 only; 

G - General Business only; and 

H - No mathematics courses; 

Other independent variables exarainod were: grade level, racial 
composition of the school, curriculum type, achievement level, race/ethnicl ty , 
and affective measures. The dependent variable was the mean percentage of 
correct responses to the algebra or pre-^algebra subtests* Also, in the 
affective domain, the dependent variable waiR the weighted mean proportion of 
responses to Likert-type affective exercises. 
Analysis of the Data 

The NAEP sampling technique Involved the selection of groups of students 
from the same school; thus, observations were not Independent of each other. 
A procedure developed by The Education Commission of the States was used to 
adjust ordinary formulas based on the assumption of simple random sampling 
(Johnson 1980). Significance tests were conducted for ^ « .01 o 
Results of the Study 
Inequitable Access 

Significant racial/ethnic differences in enrollment by curriculum were 
found. Blacks were slightly more likely than whites (47% versus 43%) to be 
enrolled in the general curriculum. However, white students were substantially 
more likely than Black students (46% vs 36%) to be enrolled in the academic 
curriculum. It is of interest to note that among students enrolled in the 
academic curriculum, 55% of the Blacks, compared to 79% of the whites, were 
enrolled in advanced aiathematics courses (Sequences A - D). As shown in Figure 
1, fewer than 12% of this number were enrolled in Sequence A, the highest and 
most advanced level. Of the students within the academic curriculum who had 
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not studied any high school mathematics, 32% were Black. 

Figure 1 Here 



Differential Mathematics Proflclenc>r 

By concepts and courses . Even though persistence In the study of 
advanced mathematics Increased proficiency in pre-algebra and algebra, 
regardless of race, significant racial differences In achievement were found. 
For example, the pre-algebra test measured: the recall of operations with 
Integers, decimals, and fractions; knowledge and understanding of 
measurement-related concepts; skills in problem solving, geometry, 
probability, and the interpretation of graphs and charts. Outcomes on this 
test revealed a large and significant racial difference of 19%. This 
difference represented nearly two standard devlatlonal units in the 
pre-algebra proficiency of white students over Blacks. The algebra test 
consisted of elementary concepts generally taught at the ninth grade level. 
The large and significant achievement gap of 17% on the algebra test 
represented a difference of one standard deviation In the proficiency of 
whites over Blacks. 

As shown in Figure 2, racial differences in achievement on the 
Total Test (the combined pre-algebra and algebra test) increased as the 
difficulty level of the courses increased. In standard deviation units, this 
differential was 1.0 at Sequence D and Increased to 1.5 for students enrolled 
at the level of Sequence A. 

Figure 2 Here 
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FIGURE - *2 

Performance on Total Test by Race/Ethnicity 
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By Qchools . Attending predominantly black high schools promoted 
slightly higher^ although not significant, increases in the enrollment of 
Black students in advanced mathematics courses. But attending predominantly 
white high schools increased the achievement outcomes for whites and Blacks 
at levels significantly higher than their racial counterparts in predominantly 
black high schools. 
Related Findings 

The author is currently analyzing items of the NAEP mathematics 
assessment to examine more specifically the problems encountered by Blacks. 
Preliminary findings show a pattern of reduced proficiency on items containing 
multiple parts. Even when the arithmetical operation remained the same, the 
majority of Black students were unable to complete items after the second 
problem of a multiple-part item at a 70% proficiency level or better. Speed 
was not a factor. Proficiency was low, too, on problems which: a) required the 
interpretation of graphs and charts; b) required the development or use of a 
model; c) req^aired knowledge of formulas, and d) involved two-step problem 
solving. 

The existence, then, of disproportionately more NAEP mathematics items 
for which the r-biserial correlational measure is less than 30 for Blacks 
compared to whites, is not surprising (NAEP normally uses this measure to 
eliminate "unfair*' items). Despite this approach to statistically assuring a 
pool of unbiased items, outcomes for Blacks showed that 9% to 17% (compared to 
3% average for whites) of the items within a given test booklet had 
r-biserials below 30. These items are generally characterized by their use 
of the term, "average." It obviously constitutes a word bias for Black 
students. This finding suggests the need to use additional criteria in 
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determining lUera bias for Blackfl, NAEP, though, is to be comraonded for 
progress made in eliminating racial stereotyping in item types and in 
Involving minorities in the construction of its assessment instruments. 

Discussion 

Collectively, the findings of this study suggest that the majority of 
Black youi:h are not enrolled in classes where mathematics is taught in 
academically challenging and stimulating ways. They do not, therefore, benefit 
from the use of enhanced instructional resources and experiences normally 
provided college-bound students in the academic curriculum. Still, Blacks, 
like other students, must be taught mathematics in academically challenging 
modes to better prepare them to enter mathematics-related vocations and 
careers in proportion to their representation in the general populatirn. 

Very little progress has been made in improving Black students' 
mathematical skills because research in this area has been relatively scarce 
and problematic. The fundamental problem of how to identify minority groups 
has yet to be resolved. In fact, most researchers are rarely certain whether 
the group is being defined by test scores, grade-level achievement, 
soci-economlc status, or other measures. To the contrary, the literature on 
gender differences and on white students and mathematics has been 
systematically developed. It is no surprise, then, that mathematics reform has 
been historically normed on the well established research base on white 
students. 

Chartered by the federal government to monitor the educational training 
of America's youth, NAEP could provide a valuable database for researching the 
mathematics proficieny of Black youth. Unlike the Scholastic Aptitude Test 
which is administered to the most capable students, NAEP is an achievement 



11 



7 

test which Is administered to "typical" In-school populations; Its 
sampling of minority generally exceeds tholr representation In the general 
population. Moreover, the numerous replications of the NAEP design at 
state levels suggest the appropriateness of NAEP for making substantial 
contributions to educational policy, particularly policy governing the 
mathematics education of racial minorities. Thuo , it is strongly recommended 
that NAEP broaden its assessment scope to Include measures of the opportunity 
to learn mathematics as a standard variable. These measures should be Included 
In a common core, and not a rotated battery to ensure adequate sample sizes. 

NAEP should also broaden its reporting format. The traditional NAEP 
explanation of score differences on tie basis of socio-economic status, 
race/ethnicity, and cognitive development tiues not readily translate into 
policy-regulating actions which can lead to programmatic changes. In addition 
to reporting achievement in the absolute, NAEP should also report achievement 
as impacted by such factors as school culture, classroom processes, and 
significant others. Until equity issues (in this case issues regarding the 
opportunity to learn mathematics) have been appropriately built into 
large-scaled instruments such as the NAEP, this nation can not benefit 
from a well-developed research base on the mathematics education of Blacks* 
Neither increased mathe^aiatics proficiency among Blacks, nor a sizeable 
decrease in the Black/white achievement gap in mathematics will likely occur. 



Proposals for Change 

Cognitive iLc^ms 

As discussed previously, many Black students failed to complete all 
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components In multiple-part assessment items. They also performed poorly on 
measurement items requiring the recall of formulas. NAEP could restructure 
cognitive items in the Interest of exploring possible causes for these 
response patterns. To determine the influence, for example, of the length of 
one's attention span on proficiency as measured by problems presented in 
multiple-item formats, the use of "matched" items is recommended. Such 
built-in controls could' also promote the understanding of the influence of 
value judgement (such as the perception that materials easily retrievable from 
reference sources should not be committed to memory) on student achievement. 

Providing measurement formulas in selected instances and assessing 
skills when these formulas are not provided would facilitate a better 
understanding of students* capabilities. Questions regarding reduced 
proficiency owing to controllable factors like these remain unanswered 
because NAEP does not in a given booklet, use items which would provide 
amplification of error patterns. 

It is further recommended that selected cognitive items on which Blacks 
have repeatedly failed to achieve at the level of 50% or better be entered 
in subsequent assessments with accompanying background measures which 
monitor the: 

1) students* familiarity with the content, format, and mode of 
presentation of the item; 

2) use of extra-curricular activities and/ or instructional materials to 
enhance understanding 

3) students' interests in solving the types of real-world problems 
presented. 

These recommendations merit further study. However, they represent a 
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start in a much needed direction to ascertain what happens to blacks during 
their precollege mathematics training experiences. 
Background and Affective Questions 

The content of selected background questions and the procedure for 
eliciting responses to the affective questions do not yield useful 
information on Blacks. A typical background question which should be 
revised is the index of items in the home (encyclopedias, vacumn cleaners, 
books, magazine, etc.) as an SES measure. Many middle income Blacks have 
access to, but do not have encyclopedias in their homes. Although no specific 
recommendation for a replacement item is made, further study of the type 
reading materials Black families have in their homes or have access to is 
s trongly encouraged . 

NAEP included different sets of affective measures in different booklets. 
The net result was that in a sample of 2000 students, approximately 200 of 
whom were Blacks, not enough Blacks were included to draw reasonable 
inferences on their motivations and aspirations in mathematics by racial 
composition of the school or by race and gender status. Because the affective 
items were not administered to all respondents, analyses at increasing levels 
of details were not possible. The cross tabulations either yielded empty 
cells or N's that were too small for analyses. It is important, then, to 
include affective measures on Blacks in a core battery rather than a rotated 
format to eliminate this problem. 

Proposals for improving the content of affective and/or background 
measures to better meet the needs of Black youth include the use of measures 
of the student^s perceptions of: 
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1) their teacher's expectations for them to learn mathematics; 

2) the procedures, methods, programs, etc. used by the school, teacher, 
etc. to encourage persistence in the study of mathematics; 

3) explanations of poor mathematics achievement in school (to include 

information on the number of repeated courses in mathematics); 

4) exclusionary (encouraging) tactics which instructors use to discourage 
(encourage) their participation in the mathematics classroom; and 

5) the role of their families in assisting with mathematics homework 
and in encouraging them to study mathematics. 

Answers should also be sought to the following: 

1) measures which are used for advising students into mathematics 
courses; and 

2) measures which differentiate the influence of letter grades 
from personal motivation and/or social utility jr studying 

mathematics. 

Conclusion 

Black high school students constitute an underrepresented group in 
mathematics. School-related barriers which communicate messages of " being 
poor in mathematics" work together with other forces to preclude Black 
students* access to advanced mathematics courses. Like other students, Blacks 
should be taught mathematics in academically challenging modes to better 
prepare them to enter mathematics-related vocations and careers in proportion 
to their representation in the general population. 

Barriers such as: discriminatory content in the formal and informal 
curriculum; testing and advisement practices which discriminate often by 
exclusion; and negative teacher behaviors can be addressed by policy. 
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Policy makers, however, need access to a knowledge base which Is the 
by-product of well-focused and systematically conducted research. Such access 
will promote improved decision-making capabilities in the effort to improve 
the quality of mathematics education of Black youth in America. NAEP can make 
a valuable contribution in this regard. 

NAEP should, in fact, be more integrally involved in monitoring the 
access to learning, and the in-class, as well as, in-school experiences of the 
nation's youth, given the national demands for school accountability, as 
expressed, for example, in the Nation at Risk (National Commission on 
Excellence in Education 1983). In this time of shrinking federal dollars for 
compensatory education and during this time when educators, administrators, 
curriculum supervisors, interested lay persons, and policy makers frequently 
use test data to justify such decisions as program expansion or deletion, 
student advancement or retention, and the construction or closing of school 
facilities, "fairness" in testing is an issue of major concern for Blacks. 
NAEP should, then, seek to incorporate and to clarify through subsequent 
assessments the opportunities that all students have to learn mathematics. In 
this way, policy makers and other concerned individuals will be able to use 
their powers more perceptively in judging and in creating opportunities for 
Black youth to achieve in mathematics. 
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